Despite the significant progress of HDD (hard disk drive) technology over the past years, magnetic tape systems constitute an integral part of current tiered storage infrastructures. Tape technology offers several important advantages including low-cost, long-term storage of data as well as for backup and disaster recovery purposes, energy savings, security, lifetime, and reliability.
Once data has been recorded in tape systems, the medium is passive. This means that it simply sits in a rack and no power is needed. Compared with similar disk-based systems, a tape-based archive consumes approximately 290 times less power. In terms of security, once data has been recorded and the cartridge removed from the access system, the data is inaccessible until the cartridge is reinstalled in the active system. Security is further enhanced by drive-level encryption, which was introduced in Linear Tape Open generation-4 drives (LTO-4) and is also standard in enterprise-level tape drives. The tape medium has a lifetime of 30+ years; however, this is rarely taken advantage of because of the rapid advances in tape hardware and the cost savings associated with migration to higher-capacity cartridges. In terms of reliability, LTO-4 tape has a bit error rate that is at least an order of magnitude better than that of a SAS (Serial Attached SCSI) HDD. Moreover, the fact that tape media is removable and interchangeable, means that, in contrast to HDDs, mechanical failure of a tape drive does not lead to data loss because a cartridge can simply be mounted in another drive.
All of the above advantages contribute to the major net advantages of tape system, which are cost and reliability. Estimates of cost savings between disk and tape range from a factor of three to more than 20.
Hard disks provide random access to data and generally contain a file index managed by a file system. These files can be accessed by means of standard sets of application programming interfaces (APIs) using various operating systems and applications. Tape, in contrast, is written in a linear sequential fashion typically using a technique called “shingling” which provides backward write compatibility, but also implies that new data can only be appended at the end and that previously written areas can only be reclaimed if the entire cartridge is reclaimed and rewritten. In traditional tape systems, an index of the files written on a given cartridge is usually only kept in an external database managed by an application such as a proprietary back-up application. The need to access an external database to retrieve data renders data on tape much less portable and accessible than with alternative storage methods, such as a HDD or a USB (Universal Serial Bus) drive.
To address these deficiencies, a new file system, referred to as Linear Tape File System (LTFS), has recently been introduced in the LTO-5 tape-drive systems to enable efficient access to tape using standard and familiar system tools and interfaces. LTFS is implemented by taking advantage of the dual-partition capabilities supported in the new LTO-5 format. A so-called index partition is used for writing the index, and the second, much larger partition for the data itself. The index partition is used for the directory of the file system, whereas the data partition is used to store the actual user's files in the file system. Placing the index on a separate partition is advantageous because it can be rewritten without impacting the actual data. LTFS exposes a POSIX-like file system interface to the user, manages the file system index in the index partition and stores the data in the data partition. The file system represented by the LTFS software makes files and directories show up on the desktop with a directory listing while the files are actually located on tape. File system users can “drag and drop” files to and from tape and can run applications developed for disk systems. In library mode, the content of all volumes in the library can be listed and searched without mounting the individual cartridges. All these features help reduce tape, file management and archive costs and eliminate the dependency on a middleware layer. Hence the cost per gigabyte (GB) stored is reduced. In addition, tape becomes cross-platform-portable (LTFS is available on Linux®, Apple Mac OS X®, Microsoft Windows®), enabling and facilitating the sharing of data between platforms. These features enable significant new use cases for tape, such as video archives, medical images, etc. Considering the cost advantages of tape over other storage solutions, the demonstrated potential for the continued scaling of tape-cartridge capacity and cost per GB as well as the increasing usability of tape provided by advances such as the LTFS, tape appears set to play an important role in the exploding market for archival data storage solutions.
However, even with LTFS, files are stored in a sequential manner on tape which causes non-negligible longer access times during I/O operations such as read and write.
The documents ‘Implementing an Automated Digital Video Archive based on the video edition of Xendata Software’, XenData White Paper, April 2007, US 2008/0040539 A1, US2010/0211731 A1, U.S. Pat. No. 7,864,479 B2, U.S. Pat. No. 5,829,046, U.S. Pat. No. 7,644,204 B2, U.S. Pat. No. 6,711,580 B2, ‘SW) COPANTM Virtual Tape Library—300T/TX’, ‘Driving to best practices in archiving’, L. DuBois, March 2007, Page 6, ‘The case for massive arrays of idle disks’, D. Colarelli, D. Grunwald and M. Neufeld, Dept. of Computer Science, Univ. of Colorado, Boulder, Jan. 7, 2002, Page 3, ‘Optimizing virtual tape performance: improving efficiency with disk storage system’, D. Cuddihy, Atto Technology Incorporation, Embedded Software Group, June, 2007, Page 6, are related to the field of the invention.